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1 Incorporation By Reference 

2 U. S. Patent Application Serial No. , filed on even date herewith in the names 

3 of Gideon P. Stein, Ofer Mano And Amnon Shashua, and entitled "System And Method For 

4 Estimating Ego-Motion Of A Moving Vehicle Using Successive Images Recorded Along The 

5 Vehicle's Path Of Motion" (hereinafter referred to as "the Stein patent application"), assigned to the 

6 assignee of the present application, incorporated herein by reference. 

7 Field Of The Invention 

j * 8 The invention relates generally to the field of systems and methods for estimating time-to- 

H 9 contact between a moving vehicle and an obstacle and more specifically to systems and methods that 

I L |10 estimate time-to-contact using successively-recorded images recorded along the vehicle's path of 

/Jfll motion. 

! *i 2 Background Of The Invention 

J^ 1 3 Accurate estimation of the time-to-contact between a vehicle and obstacles is an important 

pl4 component in autonomous driving and computer vision-based driving assistance. Using computer 

1 5 vision techniques to provide assistance while driving, instead of mechanical sensors, allows for the 

16 use of the information that is recorded for use in estimating vehicle movement to also be used in 

17 estimating ego-motion identifying lanes and the like, without the need for calibration between 

18 sensors as would be necessary with mechanical sensors. This can reduce the cost of the 

1 9 arrangements provided to provide time-to-contact estimates and maintenance that may be required 

20 therefor. 
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1 Summary Of The Invention 

2 The invention provides a new and improved system and method that estimates time-to- 

3 contact of a vehicle with an obstacle using successively-recorded images recorded along the vehicle's 

4 path of motion. 

5 In brief summary, the inventio provides a time-to-contact estimate determination system for 

6 generating an estimate as to the time-to-contact of a vehicle moving along a roadway with an 

7 obstacle. The time-to-contact estimate determination system comprises an image receiver and a 

8 processor. The image receiver is configured to receive image information relating to a series of at 

9 least two images recorded as the vehicle moves along a roadway. The processor is configured to 
jl 0 process the image information received by the image receiver to generate a time-to-contact estimate 
*jl 1 of the vehicle with the obstacle. 

rji 2 Brief Description Of The Drawings 

| i: 

*13 This invention is pointed out with particularity in the appended claims. The above and 

L|14 further advantages of this invention may be better understood by referring to the following 

L;jl5 description taken in conjunction with the accompanying drawings, in which: 

1 6 FIG. 1 schematically depicts a vehicle moving on a roadway and including a time-to-contact 

1 7 estimation system constructed in accordance with the invention; 

1 8 FIGS. 2 A and 2B are useful in understanding the apparent horizontal motion of an obstacle 

19 in a series of images as a function of horizontal position of the obstacle relative to the vehicle, which 

20 is useful in estimating the likelihood of the vehicle contacting the obstacle; and 

21 FIG. 3 is a flow chart depicting operations performed by the time-to-contact estimation 

22 system in estimating time-to-contact of the vehicle with an obstacle. 
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1 Detailed Description of an Illustrative Embodiment 

2 FIG. 1 schematically depicts a vehicle 1 0 moving on a roadway 1 1 and including a time-to- 

3 contact estimation system 12 constructed in accordance with the invention. The vehicle 10 may be 

4 any kind of vehicle 10 that may move on the roadway 1 1 , including, but not limited to automobiles, 

5 trucks, buses and the like. The time-to-contact estimation system 12 includes a camera 13 and a 

6 processor 14. The camera 13 is mounted on the vehicle 10 and is preferably pointed in a forward 

7 direction, that is, in the direction in which the vehicle would normally move, to record successive 

8 images as the vehicle moves over the roadway. Preferably as the camera 1 3 records each image, it 

9 will provide the image to the processor 14. The processor 14, in turn, will process information that 
glO it obtains from the successive images, possibly along with other information, such as information 
*!11 from the vehicle's speedometer (not separately shown) to estimate a time-to-contact value 
yl 2 corresponding to the estimated time-to-contact, if any, of the vehicle 1 0 with one or more obstacles, 
113 generally identified by reference numeral 15. The processor 14 may also be mounted in or on the 
JJ14 vehicle 1 1 and may form part thereof The time-to-contact estimates generated by the processor 14 

1 5 may be used for a number of things, including, but not limited to autonomous driving by the vehicle, 

J. 6 providing assistance in collision avoidance, and the like. Operations performed by the processor 14 

U17 in estimating time-to-contact will be described in connection with the flow chart depicted in FIG. 

318 3. 

1 9 Before proceeding further, it would be helpful to provide some background for the operations 

20 performed by the collision detection and time-to-contact estimation processor 14 depicted in FIG. 

21 1 in detecting possible obstacles to motion of the vehicle 10 and the time period to contact between 

22 the vehicle 10 and the respective obstacles, if any. Generally, the operations performed by the 

23 processor 14 can be divided into two phases, namely, an obstacle detection phase and a time to 

24 contact estimation phase. It will be appreciated that these phases may overlap in connection with 

25 various ones of the obstacles, with the processor 14 engaging in the obstacle detection phase to 
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1 attempt to detect new obstacles while it is engaging in the time to contact estimation phase to 

2 determine time-to-contact with previously-detected obstacles. 

3 During the obstacle detection phase, two general operations are performed. Initially, using 

4 a roadway deletion operator, as between successive images *F, the portions of the images that 

5 comprise projections of the roadway are identified. The portions of the images that are identified 

6 as comprising projections of the roadway will be ignored. Using an obstacle detection operator, 

7 portions of the images other than those that comprise projections of the roadway are analyzed to 

8 identify obstacles. After the obstacles are identified, during the time to contact estimation phase, a 

9 time-to-contact estimate is generated for the identified obstacles. The time-to-contact estimate 

10 indicates whether the vehicle 10 and obstacle are moving closer together, farther apart, or 

1 1 maintaining a constant separation. In particular, if the time-to-contact estimate for a particular 

12 obstacle is positive, the vehicle 10 and the obstacle are moving closer together, if the estimate is 

1 3 infinite, the separation is constant, and if the estimate is negative, the vehicle 1 0 and the obstacle are 

14 moving farther apart. 

15 It will be appreciated that the time-to-contact estimate generated for a particular pair of 

16 images *P and W will provide an estimate as to the time-to-contact at the time that the later image 

17 *F was recorded. However, the time-to-contact estimate generated as described herein does not take 

18 into account other information that may be useful in determining the likelihood of whether the 

19 vehicle 10 will contact the obstacle if their relative motion remains constant. For example, ego- 

20 motion information (which may be provided as generated, for example, using a methodology 

21 described in the Stein patent application) and information as to the shape of the roadway and 

22 expected path of the vehicle may be useful in determining whether the vehicle will likely contact the 

23 obstacle. If, for example, the obstacle is in the same lane as the vehicle 1 0, and if the time-to-contact 

24 estimate is positive, indicating that the separation of the vehicle 1 0 and obstacle is decreasing, it may 

25 be determined that the vehicle 10 will likely contact the obstacle if the vehicle's motion is not 

26 changed. In addition, using the same information, strategies may be developed for avoiding contact, 

27 such as changing speed changing lanes or other strategies as will be apparent to those skilled in the 

28 art. 
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1 As noted above, during the obstacle detection phase, a road detection operator and an 

2 obstacle detection operator are used to facilitate identification of obstacles. Although the obstacle 

3 detection operator alone could be used in identification of obstacles, since the roadway is not an 

4 obstacle, and since typically the projection of the roadway typically comprises relatively large 

5 portions of the respective images *F and those portions can be ignored in applying the obstacle 

6 detection operator. The roadway detection operator initially tessellates, into corresponding regions 

7 "R. M the image ¥ and an image ¥ ' , where image ¥ ! is the warp of image *F toward image ¥ using 

8 the estimated motion of the vehicle between the time at which image *F was recorded to the time at 

9 which image *F is recorded, and generates values "Q" as follows: 

Q - E(*'-*) 2 ay 

x,yeR 

Ml 1 The estimated motion that is used to generate the warped image ¥ 1 comprises the translation and 

t Si 2 rotation of the vehicle 1 0 as between the point in time at which image ¥ was recorded and the point 

■ [if 13 in time at which image was recorded, and may be an initial guess based on, for example, the 

j.&14 vehicle's speed as provided by a speedometer, or the estimated motion as generated as described in 

jyl5 the Stein patent application. It will be appreciated that the warped image ¥ 1 generally reflects an 

pi 6 estimate of what the image would have been at the time that the image ¥ was recorded. This holds 

* rf 17 for regions that are projections of the roadway, but not necessarily for other regions, and particular 

18 will not hold for regions that have a vertical extent, which will be the case for obstacles. 

1 9 Accordingly, regions in images ¥ and ¥ ' for which the value "Q" is below a selected threshold will 

20 be deemed to be regions that are projections of the roadway, and other regions will be deemed not 

21 to be regions that are proj ections of the roadway. Similarly, regions in image which were warped 

22 to regions in image ¥ ' that are deemed to be regions that are projections of the roadway, are also 

23 deemed regions that are projections of the roadway, and other regions in image *F f are deemed not 

24 to be projections of the roadway. 
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1 After the roadway detection operator has been used to identify regions in images *P and 

2 that are not projections of the roadway, those regions are processed using the obstacle detection 

3 operator to detect obstacles in those regions. Generally, it will be appreciated that, unlike the 

4 roadway or elements, artifacts such as roadway markings, shadows cast on the roadway surface, and 

5 the like, all of which have only a horizontal extent, obstacles have a vertical extent as well as a 

6 horizontal extent. Accordingly, as between two successive images Y and W 1 , artifacts such as 

7 markings on the roadway surface, shadows cast on the roadway surface, and the like, will exhibit 

8 expansion along the images' horizontal axes and little or no expansion along the vertical axes. On 

9 the other hand, obstacles, which have vertical extent, will exhibit more uniform expansion along the 

10 images' vertical axes as well as along the horizontal axes. 

1 1 With this observation, in the obstacle detection phase, the images are processed using two 

1 2 filters are used, namely, a roadway filter and an obstacle filter. The roadway filter filters out portions 

13 the roadway and associated artifacts on the images Y and *F, and leaves the obstacles. The affme 

14 motion of a patch as between image Y and image *F that is a projection of an obstacle exhibits a 

1 5 generally uniform scaling as around a "focus of expansion" ("FOE") for the obstacle. That is, if the 

16 FOE of an obstacle is at point p FO E( x o>yo) m the image Y, the motion vector (u,v) of a projection of 

17 a point P(X,Y,Z) on the obstacle, which projects onto point p(x,y) in the image *F, will have 

18 components 

^ u - s(x- x^) (2) 

20 v = s(y-y 0 ) (3) 

21 where "s" is a constant that reflects the uniform scaling. The components of the motion vector 

22 indicate the horizontal and vertical motion of a proj ection of a point P(X 7 Y, Z) in three dimensional 
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1 space as between the projection p(x,y) in image ¥ and the projection p'(x',y') in image If the 

2 coordinates of the FOE are not known, the motion vector will still reflect a uniform scaling, and will 

3 also reflect a translation 



u = sx + x 



f 



(4) 



v = 



sy+ y f 



(5) 



!tal 6 

H 7 
SlO 



where Xf= -sx 0 and -sy 0 . It will be appreciated that, in equations (4) and (5), the "sx" and "sy" 
terms reflect the uniform scaling, and the x f and y f terms reflect the translation. Since, for the motion 
vector, the components u=x'-x and v=y'-y, where "x" and "y" are the coordinates of the projection 
of a point in three-dimensional space in image ¥ and "x"' and "y"' are the coordinates of the 
projection of the same point in three-dimensional space in image and so 



1-11 

i y 

s ' f-"i 

i;5i2 

13 



(5+1) 0 
0 (5+1) 



+ 



\ -yj 



Substituting equation (6) into the constant-brightness criterion 



ul x + vl + I t = 0 



(6). 



(7), 



14 
15 



yields 



(sx+x f )l x + (sy+y f )l y 



= -I 



(8), 
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1 

2 



which can be written 



V ( \ 

yj 

\ s J 



(9), 



3 where the superscript "T" represents the transpose operation. Regions of images *F for which 

4 equation (9) hold represent obstacles. 

5 If one or more obstacles is detected during the obstacle detection phase, the processor 1 4 can 

6 generate an estimate of the time-to-contact, if any, of the vehicle 1 0 with the obstacle. Although the 
^ 7 time-to-contact will reflect the distance between the vehicle and the obstacle, it will be appreciated 
3 8 that in many circumstances the time-to-contact is a more useful metric. The distance of the obstacle 
y 9 to the image plane of the camera 1 3 , which, in turn, reflects the distance of the obstacle to the vehicle 
y l 0 1 0, at the point in time T-AT at which image *F was recorded is Z+AZ, and the distance at the point 
rfl 1 in time T at which image was recorded is Z. (It should be noted that the values of both "AT" and 
" : 1 2 "AZ" are positive). If there is no change in relative motion between the vehicle 1 0 and the obstacle, 
*1 3 the distance Z to the obstacle will be closed in a time period T, that is, the time-to-contact is T. 

SI 4 The processor 1 0 estimates the time-to-contact T in relation to the ratio of the scaling of the 

31 5 obstacle as recorded in the images *F and if any, and specifically in relation to the scaling of the 

16 vertical dimension. The vertical dimension of the obstacle in image *F is 

17 



y 2 = 



Z+ AZ 



(10) 



18 
19 

20 



where "Y" refers to the height of the obstacle in three-dimensional space and "f is the focal length 
of the camera 13. Similarly, the vertical dimension of the obstacle in image is 



y\ = 



Z 



(11), 
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1 since the height " Y" of the obstacle in three-dimensional space does not change as between the times 

2 at which the images Y and *F are recorded. The scaling factor, or ratio, of the vertical dimensions 

3 H S" in the images is 

y x Z+AZ AZ 

s - 1 = — = <i2) - 

4 

5 In addition, assuming that there is no change in motion as between the vehicle 1 0 and the obstacle, 

AZ 

6 the relative speed between the vehicle 1 0 and the obstacle during the time period between the 

Z 

7 points in time at which the images Y and were recorded, will be the same as the relative speed — 

T 

8 during the time period between the point in time at which the image *F was recorded and the point 

9 in time at which the vehicle 10 would contact the obstacle. That is 

10 AZ Z 



11 Rearranging equation 13, 

T Z 
AT = A2 

12 

1 3 Combining equation (14) and equation (12), 
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AZ AT 
S = 1+ = 1+ (15). 



Rearranging equation (15) to solve for T, the time-to-contact, 



T = -r^rAr (16). 



It will be appreciated that, if the value of AZ is positive and non-zero, from equation 12, the value 

1 

of S will be greater than one, in which case the value of the factor in equation (16) will be 

S — 1 

greater than zero. In that case, the time-to-collision T will be positive. It will be appreciated that 
this can occur if the distance separating the vehicle 10 and the obstacle is decreasing. On the other 
hand, if the value of the ratio S is equal to one, which can occur if the vertical dimension of the 

1 



obstacle in the image ¥ is the same as the vertical dimension in the image *F, the value of the factor 



S- 1 



in equation (16) is infinite, in which case the time-to-contact T will also be infinite. It will be 
appreciated that, when that occurs, if the vehicle 10 is moving, the obstacle will also be moving, and 
it will be moving at the same speed as the vehicle 10, and in that case, the separation between the 
vehicle 10 and the obstacle will be constant. Finally, if the value of the ratio S is less than one, 
which can occur if the vertical dimension of the obstacle in the image *F is the greater than the 

vertical dimension in the image the value of the factor — - — in equation (16) is negative, in 

S ~ 1 

which case the time-to-contact T will also be negative. It will be appreciated that, when that occurs, 
the obstacle will also be moving, and it will be moving at the a speed that is greater than, and in a 
direction away from, the vehicle 10. It will be appreciated that, using equation (16), the processor 
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1 14 can estimate the time-to-contact using only information from the images ¥ and *F and the time 

2 period AT, without needing any information as to the actual distance between the vehicle 1 0 and the 

3 obstacle. 

4 As noted above, the time-to-contact value generated as described above actually reflects the 

5 rate at which the separation between the vehicle 1 0 and the obstacle is decreasing at the point in time 

6 that the image is recorded. Depending on the particular positions and velocities (which reflect 

7 direction as well as speed) of the vehicle 1 0 and the obstacle at any point in time, as well as the sizes, 

8 and primarily the horizontal dimensions, of the vehicle 1 0 and the obstacle, the vehicle and obstacle 

9 may or may not actually come into contact. This will be described in connection with FIGs. 2A and 
10 2B. With reference to FIG. 2A, that FIG. schematically depicts a portion of a curved roadway 20 

J 1 with three lanes, including a center lane represented by line 21 C, a lane to the left of the center lane 

31 2 represented by line 2 1 L and a lane to the right of the center lane represented by line 2 1 R. The center 

yjl 3 of curvature of the roadway is at O. The vehicle 1 0 is traveling in the center lane 2 1 C, and obstacles, 

^14 which comprise other vehicles, are traveling in all three lanes. To simplify the explanation, the 

ril 5 coordinate system will be deemed to be moving with the vehicle 1 0. It will also be assumed that a 

: 16 series of images are recorded, including a first image T when the obstacles are at locations a, a', a", 

*1 7 a second image when the obstacles are at locations b, b', b M , a third image V F" T when the obstacles 

yl 8 are at locations c, c r , c", a fourth image when the obstacles are at locations d, d\ d", and so forth. 

1:19 In this example, the separations between vehicle 10 and all three obstacles are decreasing. Graphs 

1120 of the separations and the horizontal coordinates of the respective obstacles in the series of images 

2 1 (it will be appreciated that the vertical coordinates of the respective obstacles in the images will not 

22 change) are depicted in FIG. 2B, with the separation with the obstacle in lane 21 C being represented 

23 by solid circles, the separation with the obstacle in lane 21L being represented by hollow circles, and 

24 the separation with the obstacle in lane 21R being represented by crosses ("+"). In FIG. 2B, the 

25 vehicle 1 0 will be considered to be located at the origin, that is, where the "x-coordinate" is zero and 

26 the Z-coordinate is zero, although it will be appreciated that, because the vehicle 1 0 will have a non- 
27 zero horizontal dimension, the portion of the horizontal dimension of the width subtended by the 
28 vehicle extends to the left and right of the origin by an amount associated with the width of the 
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1 vehicle. As shown in FIG. 2B, all three obstacles will initially approach the vehicle 1 0, but at some 

2 point the separations between vehicle 10 and the obstacles in the left and right lanes 21R and 21L 

3 will again increase. However, the separation between obstacle in the same lane 21C as that of the 

4 vehicle 10 will continue to decrease. If the time intervals between times at which the images are 

5 recorded are uniform the horizontal coordinate of the projection of the obstacle in lane 21 C will 

6 uniformly approach zero. For obstacles in the other lanes 21L and 22R, the movement of the 

7 horizontal coordinate will not be uniform. Using the information as to the uniformity of the 

8 progression of the horizontal coordinates of the projection of the vehicles across the successive 

9 images, the likelihood as to whether the vehicle will contact an obstacle can be determined. 

10 With this background, operations performed by the processor 14 in connection with 

^1 1 determining the time-to-contact will be described in connection with the flow chart depicted in FIG. 

312 3. With reference to FIG. 3, after receiving anew image *F, the processor 14 initially estimates the 

ijl 3 ego-motion of the vehicle 1 0 (step 300) using, for example, a methodology described in connection 

*1 4 with the Stein patent application. Using the estimated ego-motion and lane information, if available, 

rll 5 the processor 14 extrapolates the future path of the vehicle (step 301) and identifies a " danger zone" 

1 6 along the extrapolated future path (step 3 02). In performing step 3 02, the processor 1 4 can make use 

H 7 of geometric assumptions that are based on the camera's calibration parameters, and the danger zone 

yl 8 can, for example, comprise a trapezoidal region along the image of the roadway. The processor 14 

'^19 then applies the roadway detection operator (step 303) and obstacle detection operator (step 304) to 

51? 

320 those portions to identify obstacles that are in or near the extrapolated path. Thereafter, if the 

21 processor 14 will determine whether there were any obstacles identified after application of the 

22 obstacle detection operator (step 305), and, if not return to step 300 to receive the next image. 

23 On the other hand, if the processor 1 4 determines in step 3 05 that one or more obstacles have 

24 been identified in step 304, the processor 14 will examine the next few images to 

25 determine whether the images contain projections of the respective obstacle(s) (step 306). Tracking 

26 obstacle(s) whose projections have been detected in one or two images through successive images 

27 provides verification that the obstacles in fact exist and are not artifacts in the respective images. 

28 For any obstacles whose projections are recorded in a predetermined number of subsequent images, 
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1 the processor 14 will determine that the obstacle(s) have been verified (step 307), and, for any 

2 verified obstacles, the processor 14 will determine the value of the scaling factor "S" for the obstacle 

3 as between successive images (step 308). After determining the value of the scaling factor, the 

4 processor 14 uses that value and the value AT in equation (16) to determine the time-to-contact T 

5 (step 309). Thereafter, the processor can make use of the time-to-contact value in connection with, 

6 for example, providing in collision avoidance assistance to the vehicle or driver, if, for example, the 

7 time-to-contact value is positive. As noted above, if the time-to-contact value is not positive, there 

8 will be no contact between the vehicle 10 and the obstacle. 

9 The invention provides a number of advantages. In particular, the invention provides an 
10 arrangement for estimating the time-to-contact of a vehicle 10 with an obstacle directly from 

J 1 successively-recorded images of the obstacle, without requiring other mechanical or electronic 

312 sensors, which can be expensive to install and maintain. 

*1 3 It will be appreciated that a number of changes and modifications may be made to the time- 

J14 to-contact estimation system 12 described above. For example, instead of using the scaling of the 

rjl5 vertical dimension of obstacles as between image Y and the system 12 may use the scaling of 

16 the horizontal dimension, or both. 

yl7 In addition, it will be appreciated that images Y and X P* are preferably rectified as, for 

fx 

;18 example, described in the Stein patent application, so that their image planes are perpendicular to 

31 9 the roadway plane. 

20 Furthermore, it will be appreciated that roadway detection filters and obstacle detection 

21 filters can be used other than or in addition to those described herein. For example, a filter such as 

22 one described in the aforementioned Stein patent application can be used to identify patches that 

23 contain projections of obstacles. 

24 It will be appreciated that a system in accordance with the invention can be constructed in 

25 whole or in part from special purpose hardware or a general purpose computer system, or any 

26 combination thereof, any portion of which may be controlled by a suitable program. Any program 
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1 may in whole or in part comprise part of or be stored on the system in a conventional manner, or it 

2 may in whole or in part be provided in to the system over a network or other mechanism for 

3 transferring information in a conventional manner. In addition, it will be appreciated that the system 

4 may be operated and/or otherwise controlled by means of information provided by an operator using 

5 operator input elements (not shown) which may be connected directly to the system or which may 

6 transfer the information to the system over a network or other mechanism for transferring 

7 information in a conventional manner. 

8 The foregoing description has been limited to a specific embodiment of this invention. It will 

9 be apparent, however, that various variations and modifications may be made to the invention, with 

10 the attainment of some or all of the advantages of the invention. It is the object of the appended 

1 1 claims to cover these and such other variations and modifications as come within the true spirit and 

12 scope of the invention. 

13 What is claimed as new and desired to be secured by Letters Patent of the United States is: 
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Claims 

1 1 . A time-to-contact estimate determination system for generating an estimate as to the time-to- 

2 contact of a vehicle moving along a roadway with an obstacle comprising: 

3 A. an image receiver configured to receive image information relating to a series of at least two 

4 images recorded as the vehicle moves along a roadway; and 

5 B. a processor configured to process the image information received by the image receiver to 

6 generate a time-to-contact estimate of the vehicle with the obstacle. 
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2 A time-to-contact estimate determination system is disclosed for generating an estimate as 

3 to the time-to-contact of a vehicle moving along a roadway with an obstacle. The time-to-contact 

4 estimate determination system comprises an image receiver and a processor. The image receiver is 

5 configured to receive image information relating to a series of at least two images recorded as the 

6 vehicle moves along a roadway. The processor is configured to process the image information 

7 received by the image receiver to generate a time-to-contact estimate of the vehicle with the obstacle. 
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FIG. 3 



300. AFTER RECEIVING A NEW IMAGE, PROCESSOR 14 
INITIALLY ESTIMATES THE EGO-MOTION OF THE VEHICLE 



} 



301. USING THE ESTIMATED EGO-MOTION AND LANE 
INFORMATION, IF AVAILABLE, THE PROCESSOR 14 
EXTRAPOLATES THE FUTURE PATH OF THE VEHICLE 



302. PROCESSOR 14 IDENTIFIES A "DANGER ZONE" 
ALONG THE EXTRAPOLATED FUTURE PATH 



J 



303. PROCESSOR 14 THEN APPLIES A ROADWAY 
DETECTION OPERATOR TO THE IMAGE 



304. PROCESSOR 14 APPLIES A OBSTACLE DETECTION 
OPERATOR TO THOSE PORTIONS TO IDENTIFY 
OBSTACLES THAT ARE IN OR NEAR THE EXTRAPOLATED 
PATH. 



305. PROCESSOR 14 DETERMINES WHETHER THERE 
WERE ANY OBSTACLES IDENTIFIED AFTER APPLICATION 
OF THE OBSTACLE DETECTION OPERATOR 



>NJ— NO- 



YES 



0 



0 



306. PROCESSOR 14 EXAMINES THE NEXT FEW IMAGES 
TO DETERMINE WHETHER THE IMAGES CONTAIN 
PROJECTIONS OF THE RESPECTIVE OBSTACLE(S) 
IDENTIFIED IN STEP 305 



-NO- 



YES 

1 



307. PROCESSOR 14 IDENTIFIES THAT THE OBSTACLE(S) 
HAVE BEEN VERIFIED IN STEP 306 



308. FOR VERIFIED OBSTACLES, PROCESSOR 14 
DETERMINES THE VALUE OF THE SCALING FACTOR "S' 
FOR THE RESPECTIVE OBSTACLE AS BETWEEN 
SUCCESSIVE IMAGES 




309. FOR EACH IDENTIFIED OBSTACLE, PROCESSOR 14 
USES THE SCALING FACTOR AND THE TIME BETWEEN 
SUCCESSIVE IMAGES ARE RECORDED TO DETERMINE 
THE TIME-TO-CONTACT T (EQUATION (16)) 



r 310. PROCESSOR 14 CAN MAKE USE OF THE ^ 
TIME-TO-CONTACT VALUE IN CONNECTION WITH, FOR 
EXAMPLE, PROVIDING IN COLLISION AVOIDANCE 
ASSISTANCE TO THE VEHICLE OR DRIVER AS 
^APP^OPRIATE^^^^^^^^^^^^^^^^^^ 



FIG.3A 



